Avatar motion modification

ABSTRACT

A method, system and computer program for modifying avatar motion. The method includes receiving an input motion, determining an input motion model for the input motion sequence, and modifying an avatar motion model associated with the stored avatar to approximate the input motion model for the input motion sequence when the avatar motion model does not approximate the input motion model. The stored avatar is presented after the avatar motion model associated with the stored avatar is modified to approximate the input motion model for the input motion sequence.

BACKGROUND

The present invention relates to avatar manipulation, and moreparticularly to a method and system for modifying avatar motion.

Today, virtual worlds, 3-D Internet, massive multi-player games,character animation in movies, telepresence, virtual meetings, andrelated are becoming more popular and important. 3-D interactive gamesare becoming increasingly popular. People may conduct business invirtual worlds. Meetings may sometimes represent attendees using avatarrepresentations. Teachers in the form of avatars may interact withstudents.

Similarly, given today's environment of expensive travel and increasedcost-cutting, more businesses are making transactions using thetelephone, using avatars in virtual meeting rooms, and via other remotemethods rather than face-to-face meetings. It is therefore desirable tomake the best impression in these remote communications, since new formsof communication are becoming common modes of performing business, andindividuals need to create impressions given access only to multimediachannels.

Similarly, novice users of avatars in virtual worlds (e.g., Second Life)and 3-D games may need help in moving their avatars and performinggestures. Beginners often do not know how to control their avatars toprovide various nuances. Also, some gestures may have different meaningsfor different viewers of the gestures.

Similarly, increasingly more characters in movies are actually based onavatars that are controlled by a human actor. Such avatars may havesubtle motions, methods of running, gestures, etc. On any given day, atany particular point during the day, a controller of an avatar might notbe in best form. A user may be tired, inexperienced, or have motordeficits. A user of an avatar may desire to make a convincing salespitch or compelling presentation, but cannot naturally and easilyexhibit the level of motion enthusiasm and nuance that he/she would wantin order to appear authoritative, energetic, contrite, etc. Some usersmay be unable to attain the broad gestural range that is needed in aparticular setting, due to disabilities that include poor motor controlor autism of various degrees.

Alternatives to avatar use include corresponding through text, and usingtextual cues to indicate emotion, energy, contrition, etc. However text(or traditional phone calls) is not always the ideal channel to use toconduct business. Another option involves face-to-face meetings, whereother characteristics (affect, gestures, etc.) can be made use of tomake strong and/or subtle points. However, as mentioned, face-to-facemeetings are not always logistically possible.

BRIEF SUMMARY

Accordingly, one example of the present invention is a method formodifying avatar motion. The method includes receiving an input motionsequence. The method further includes determining an input motion modelfor the input motion sequence. The method further includes modifying anavatar motion model associated with a stored avatar to approximate theinput motion model for the input motion sequence when the avatar motionmodel does not approximate the input motion model. Additionally, themethod includes presenting the stored avatar after the avatar motionmodel associated with the stored avatar is modified to approximate theinput motion model for the input motion sequence.

Another example of the present invention is a system for modifyingavatar motion. The system includes a system memory and a computerprocessor coupled to the system memory. The system further includes areceiving unit coupled to the computer processor. The receiving unitreceives an input motion sequence. The system further includes adetermining unit to determine an input motion model for the input motionsequence. The system further includes a modifying unit to modify anavatar motion model associated with a stored avatar to approximate theinput motion model for the input motion sequence, when the avatar motionmodel does not approximate the input motion model. The system furtherincludes a presenting unit to present the stored avatar after the avatarmotion model associated with the stored avatar is modified toapproximate the input motion model for the input motion sequence.

Yet another example embodiment of the present invention is a computerprogram product for modifying avatar motion. The computer programproduct includes computer readable program code configured to: receivean input motion sequence; determine an input motion model for the inputmotion sequence; modify an avatar motion model associated with a storedavatar to approximate the input motion model for the input motionsequence when the avatar motion model does not approximate the inputmotion model; and present the stored avatar after the avatar motionmodel associated with the stored avatar is modified to approximate theinput motion model for the input motion sequence.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter which is regarded as the invention is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other objects, features, andadvantages of the invention are apparent from the following detaileddescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1 shows a system for modifying avatar motion according to oneembodiment of the present invention.

FIG. 2 shows motion capture of an individual according to one embodimentof the present invention.

FIG. 3 shows gestures/movements embedded in a three dimensional spaceaccording to one embodiment of the present invention.

FIG. 4 shows a method for modifying avatar motion in accordance with oneembodiment of the present invention.

DETAILED DESCRIPTION

The present invention is described with reference to embodiments of theinvention. Throughout the description of the invention reference is madeto FIGS. 1-4. When referring to the figures, like structures andelements shown throughout are indicated with like reference numerals.

FIG. 1 shows a system 102 for modifying avatar motion according to oneembodiment of the present invention. The system may include a systemmemory 104, a computer processor 106, a receiving unit 108, adetermining unit 112, a modifying unit 116, and a presenting unit 122.

The receiving unit 108 is coupled to the computer processor 106 and thereceiving unit 108 receives an input motion sequence 110. Thedetermining unit 112 determines an input motion model 114 for the inputmotion sequence 110.

The modifying unit 116 modifies an avatar motion model 118 associatedwith a stored avatar 120 to approximate the input motion model 114 forthe input motion sequence 110 when the avatar motion model 118 does notapproximate the input motion model 114.

The presenting unit 122 presents the stored avatar 120 after the avatarmotion model 118 associated with the stored avatar 120 may be modifiedto approximate the input motion model 114 for the input motion sequence110.

In one embodiment of the invention, the computer processor 106 may beconfigured to select an input motion sequence 110 that is associatedwith a previously obtained input motion model 114. The input motionmodel 114 may model motion based on gestures, walking motions, facialmotions, flying motions, hand motions and/or body motions. The inputmotion model 114 for the input motion sequence may be based on a smallerset of motion variables than the input motion sequence 110.

In one embodiment of the invention, the computer processor 106 may beconfigured to present different stored avatars 120 for different viewers124 of the avatar motion. The presentation of the stored avatar 120 maybe controlled, in real-time, by changing characteristics of thecomputing device and network.

In one embodiment, the computer processor 106 may further comprise aselecting unit 126. The searching unit 126 searches an avatar datarepository 128 for the stored avatar 120 using the input motion model114.

In one embodiment, the determining unit 112 may be further configured todetermine a similarity measure between the input motion model and motionmodels of stored avatars in the avatar data repository, and select thestored avatar having the associated avatar motion model that is mostsimilar to the input motion model using the similarity measure. Thedetermining unit 112 may be further configured to restrict the search ofthe data repository based on metadata associated the stored avatar.

A user may input a motion capture file as shown in FIG. 2. The systemmay then analyze these measures using a dimensionality reductionalgorithm such as Principal Components Analysis, and assign it to acluster in the motion preference library. In this way, a large number ofmovement-types can be searched automatically to approximate the desiredmeasures (for example if the user wants the avatar to walk in the sameway she does).

FIG. 3 shows gestures/movements embedded in a three dimensional spaceaccording to one embodiment of the present invention. The movementsgraphed include jump-in-place-on-two-legs (“pjump”) and jumping-jack(“jack”). The system can make use of dimensionality reductiontechniques, specifically techniques that aim to project thehigh-dimensional set of movement measurements into a low-dimensionalspace of a constrained physical plant (i.e. the motor plant, which isthe part(s) of the body that actually move) and the commands thatcontrol it. This way, the lower dimensional model of the biomechanicalapparatus responsible for generating the gestures/movements, as well asof the potentially lower-dimensional set of neural commands used tocontrol this apparatus may be used as the motion characteristicidentified in the system.

Furthermore, by modifying this apparatus, while maintaining otheraspects of the control commands, a set of new gestures/movements can bemodeled, bearing a similarity to what would be expected of the usergiven these modifications. For example, a user's movements may beanalyzed and represented by the body and motor commands that may havegenerated these movements. This representation may constitute the motioncharacteristics of the system. The body may then be modified (i.e., auser is given very long legs, a set of wings, placed in differentgravitational field, etc.) and the motion is then modified by theforward model that expands the low dimensional representation back intoa higher dimensional set of movements, given the newly introducedconstraints on the model.

Multiple segments of a motion capture may be analyzed separately andused to select by the above means a set of movements differentially, tocreate a composite movement-type for the avatar (for example, theavatar's walking behavior may be type 112.2, but its jumping behaviormay be type 252.1). Note that the motion quality of the input motion(e.g., a motion attempted using a mouse, keyboard, or 3-D motion-capturedevice) may comprise a perceivable mood or an emotion of the inputmotion/gesture (e.g., happy, sad, confident, enthusiastic, contrite,etc.). A motion quality of the input motion may comprise a perceivableintention of the input motion (e.g., question, command, sarcasm, irony,etc.).

The desired motion quality may be manually selected based on apreference of the user (e.g., avatar controller) associated with theinput motion (e.g., selectable via a user interface). The desired motionquality and characteristics may be automatically selected based on asubstantive context associated with the input motion, and adetermination as to how the output motion should appear to intended (orpossible) recipients. In one embodiment, the desired motion quality maybe automatically selected by analyzing the possible content and contextof the input motion determining a motion approximate for how the motionshould appear to achieve an objective. A motion approximate may bedetermined based on motion models previously created for the user of theavatar. Motion models may be created via background data collection(e.g., substantially transparent to the avatar controller) or viaexplicit data collection (e.g., with the user's express knowledge and/orparticipation).

FIG. 4 shows a method for modifying avatar motion 402, in accordancewith one embodiment of the present invention. The method includesreceiving step 404. During the receiving step 404, an input motionsequence may be received. The input motion sequence may be associatedwith a previously obtained input motion model. After receiving step 404is completed, the method continues to determining step 406.

At determining step 406, an input motion model may be determined for theinput motion sequence. The input motion model may model motion based ongestures, walking motions, facial motions, flying motions, hand motionsand body motions. The input motion model for the input motion sequencemay be based on a smaller set of motion variables than the input motionsequence. After determining step 406 is completed, the method continuesto searching step 408.

At searching step 408, an avatar data repository may be searched for thestored avatar using at least part of the input motion model as a searchkey. A similarity measure may be determined between the input motionmodel, or part of the input motion model, and motion models of storedavatars in the avatar data repository. The stored avatar has theassociated avatar motion model that is most similar to the input motionmodel using the similarity measure is selected. The search of the datarepository may be restricted based on metadata associated the storedavatar. After searching step 408 is completed, the method continues tomodifying step 410.

At modifying step 410, an avatar motion model associated with the storedavatar to approximate or match the input motion model for the inputmotion sequence is modified, when the avatar motion model does notapproximate or match the input motion model.

In one embodiment, user marking (e.g., via a user interface) one or moremotions may be used. The marked motions may be analyzed to determinesubsequent desired motion qualities and characteristics. Further, it mayinclude editing the content of the motion when it is determined tocontain undesirable motions. The characteristic of the motion that ismodified in the modifying step may comprise an overall set of motionsand gestures associated with the input motion.

In another example embodiment, the characteristic of the motion may bemodified prior to transmission of the motion (e.g., at user's computerterminal). In another embodiment, the characteristic of the motion maybe modified after transmission of the motion (e.g., at the intendedrecipient, or viewer, end of the network). After modifying step 410 iscompleted, the method continues to associating step 412.

At associating step 412, a different avatar motion model may beassociated to different viewers of the avatar motion. The stored avatarmay be presented after the avatar motion model associated with thestored avatar is modified. This step may include presenting the storedavatar differently to the different viewers. After associating step 412is completed, the method continues to presenting step 414.

At presenting step 414, the stored avatar may be presented after theavatar motion model associated with the stored avatar is modified toapproximate or match the input motion model for the input motionsequence. The presentation of the stored avatar may be controlled, inreal-time, by changing characteristics of the computing device andnetwork.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

What is claimed is:
 1. A method for modifying avatar motion, the methodcomprising: receiving an input motion sequence; determining an inputmotion model for the input motion sequence; modifying an avatar motionmodel associated with a stored avatar to approximate the input motionmodel for the input motion sequence when the avatar motion model doesnot approximate the input motion model; and presenting the stored avatarafter the avatar motion model associated with the stored avatar ismodified to approximate the input motion model for the input motionsequence.
 2. The method of claim 1, wherein the input motion sequence isassociated with a previously obtained input motion model.
 3. The methodof claim 1, further comprising: associating a different avatar motionmodel to different viewers of the avatar motion; and wherein presentingthe stored avatar after the avatar motion model associated with thestored avatar is modified includes presenting the stored avatar afterthe avatar motion model associated with the stored avatar is modifieddifferently to the different viewers.
 4. The method of claim 1, whereinthe input motion model models motion based on at least one of gestures,walking motions, facial motions, flying motions, hand motions and bodymotions.
 5. The method of claim 1, wherein the presentation of thestored avatar is controlled, in real-time, by changing characteristicsof the computing device and network.
 6. The method of claim 1, whereinthe input motion model for the input motion sequence is based on asmaller set of motion variables than the input motion sequence.
 7. Themethod of claim 1, further comprising searching an avatar datarepository for the stored avatar using at least part of the input motionmodel.
 8. The method of claim 7, further comprising: determining asimilarity measure between the input motion model and motion models ofstored avatars in the avatar data repository; and selecting the storedavatar using the similarity measure.
 9. The method of claim 8, furthercomprising restricting the search of the data repository based onmetadata associated the stored avatar.
 10. A system for modifying avatarmotion, the system comprising: a system memory; a computer processorcoupled to the system memory; a receiving unit coupled to the computerprocessor, the receiving unit to receive an input motion sequence; adetermining unit to determine an input motion model for the input motionsequence; a modifying unit to modify an avatar motion model associatedwith a stored avatar to approximate the input motion model for the inputmotion sequence when the avatar motion model does not approximate theinput motion model; and a presenting unit to present the stored avatarafter the avatar motion model associated with the stored avatar ismodified to approximate the input motion model for the input motionsequence.
 11. The system of claim 10, wherein the input motion sequenceis associated with a previously obtained input motion model.
 12. Thesystem of claim 10, wherein the presentation of the stored avatar isdifferent for different viewers of the avatar motion.
 13. The system ofclaim 10, wherein the input motion model models motion based on at leastone of gestures, walking motions, facial motions, flying motions, handmotions and body motions.
 14. The system of claim 10, wherein thepresentation of the stored avatar is controlled, in real-time, bychanging characteristics of the computing device and network.
 15. Thesystem of claim 10, wherein the input motion model for the input motionsequence is based on a smaller set of motion variables than the inputmotion sequence.
 16. The system of claim 10, further comprising asearching unit to search an avatar data repository for the stored avatarusing at least part of the input motion model.
 17. The system of claim16, wherein the determining unit is further configured to determine asimilarity measure between the input motion model and motion models ofstored avatars in the avatar data repository, and select the storedavatar using the similarity measure.
 18. The system of claim 17, whereinthe determining unit is further configured to restrict the search of thedata repository based on metadata associated the stored avatar.
 19. Acomputer program product for modifying avatar motion, the computerprogram product comprising; a computer readable storage medium havingcomputer readable program code embodied therewith, the computer readableprogram code configured to: receive an input motion sequence; determinean input motion model for the input motion sequence; modify an avatarmotion model associated with a stored avatar to approximate the inputmotion model for the input motion sequence when the avatar motion modeldoes not approximate the input motion model; and present the storedavatar after the avatar motion model associated with the stored avataris modified to approximate the input motion model for the input motionsequence.
 20. The computer program product of claim 19, the computerreadable program code further configured to: search an avatar datarepository for the stored avatar using the input motion model; determinea similarity measure between the input motion model and motion models ofstored avatars in the avatar data repository; select the stored avatarusing the similarity measure; and restrict the search of the datarepository based on metadata associated the stored avatar; and whereinthe input motion sequence is associated with a previously obtained inputmotion model; wherein the presentation of the stored avatar is differentfor different recipients; wherein the input motion model models motionbased on at least one of gestures, walking motions, facial motions,flying motions, hand motions and body motions; wherein the presentationof the stored avatar is controlled, in real-time, by the changingcharacteristics of the computing device and network; and wherein theinput motion model for the input motion sequence is based on a smallerset of motion variables than the input motion sequence.